Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[docs] Scheduler features #7990

Merged
merged 4 commits into from
May 28, 2024
Merged

Conversation

stevhliu
Copy link
Member

Continuation of #7817 (see comment here) that refactors scheduler features for inference to their own doc. It includes:

  • custom timesteps and sigmas showcasing AYS
  • Karras sigmas
  • rescale_betas_zero_snr

@HuggingFaceDocBuilderDev

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@stevhliu stevhliu requested a review from yiyixuxu May 20, 2024 22:08
Copy link
Collaborator

@yiyixuxu yiyixuxu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks!

@yiyixuxu
Copy link
Collaborator

cc @Beinsezii here
we are adding a general doc page for the scheduler now; anything else we should add here?

@Beinsezii
Copy link
Contributor

Beinsezii commented May 20, 2024

I feel like a section on the timestep spacings would be beneficial, especially since they're part of the same paper referenced. The paper recommends trailing which is what I and a lot of others have settled on. trailing is unique in that its mutually exclusive with steps_offset ≥ 1 as well.

@bghira
Copy link
Contributor

bghira commented May 20, 2024

a good demonstration of the current generation of models' two primary forms of residual noise would probably be a good idea though i can't think of how to integrate that. i just see it a lot and i think the community needs language to describe it with, and common solutions to try. probably for a separate doc

@stevhliu
Copy link
Member Author

Thanks for the feedback, added a new section for timestep spacing!

a good demonstration of the current generation of models' two primary forms of residual noise

Good idea, maybe we can explore this in a separate PR :)

@stevhliu stevhliu requested a review from yiyixuxu May 21, 2024 19:26
@Beinsezii
Copy link
Contributor

Maybe a "Generation Quality" doc that has a bunch of common footguns. Like using Karras sigmas on models that weren't trained for it, or turning off set_alpha_to_one/final_sigmas_type.

Also I think solver order be explored in more depth either here or another doc because the best one is highly dependent on the rest of the params. Like, if you're going run 50 steps anyways a 1st order sampler will have plenty strong enough prediction with less hallucinations. Really have to balance the steps/guidance/order for your intended effect to bring out the best image rather than just bigger number better.

@stevhliu stevhliu force-pushed the scheduler-features branch from c87ed08 to c16d56a Compare May 24, 2024 15:39
Copy link
Collaborator

@yiyixuxu yiyixuxu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks!

@yiyixuxu
Copy link
Collaborator

@Beinsezii

for this, if you are able to contribute a doc we would be so grateful!

so I think solver order be explored in more depth either here or another doc because the best one is highly dependent on the rest of the params. Like, if you're going run 50 steps anyways a 1st order sampler will have plenty strong enough prediction with less hallucinations. Really have to balance the steps/guidance/order for your intended effect to bring out the best image rather than just bigger number better.

@stevhliu stevhliu merged commit 0ab6dc0 into huggingface:main May 28, 2024
1 check passed
@stevhliu stevhliu deleted the scheduler-features branch May 28, 2024 21:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants